A Vector-Contraction Inequality for Rademacher Complexities
نویسنده
چکیده
The contraction inequality for Rademacher averages is extended to Lipschitz functions with vector-valued domains, and it is also shown that in the bounding expression the Rademacher variables can be replaced by arbitrary iid symmetric and sub-gaussian variables. Example applications are given for multi-category learning, K-means clustering and learning-to-learn.
منابع مشابه
An Inequality with Applications to Structured Sparsity and Multitask Dictionary Learning
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived. It is applied to sharpen existing and to derive novel bounds on the empirical Rademacher complexities of unit balls in various norms appearing in the context of structured sparsity and multitask dictionary learning or matrix factorization. A key role is played by the largest eigenvalue ...
متن کاملRademacher and Gaussian Complexities: Risk Bounds and Structural Results
Abstract We investigate the use of certain data-dependent estimates of the complexity of a function class, called Rademacher and Gaussian complexities. In a decision theoretic setting, we prove general risk bounds in terms of these complexities. We consider function classes that can be expressed as combinations of functions from basis classes and show how the Rademacher and Gaussian complexitie...
متن کاملLearning Theory and Algorithms for Revenue Optimization in Second-Price Auctions with Reserve A. Proofs for learning guarantees
A.2. Contraction lemma The following is a version of Talagrand’s contraction lemma (Ledoux & Talagrand, 2011). Since our definition of Rademacher complexity does not use absolute values, we give an explicit proof below. Lemma 8. Let H be a hypothesis set of functions mapping X to R and Ψ1, . . . ,Ψm, μ-Lipschitz functions for some μ > 0. Then, for any sample S of m points x1, . . . , xm ∈ X , t...
متن کامل1 99 9 the Distribution of Vector - Valued Rademacher Series
Let X = ε n x n be a Rademacher series with vector-valued coefficients. We obtain an approximate formula for the distribution of the random variable ||X|| in terms of its mean and a certain quantity derived from the K-functional of interpolation theory. Several applications of the formula are given. In [6] the second-named author calculated the distribution of a scalar Rademacher series ε n a n...
متن کاملEstimates of the Approximation Error Using Rademacher Complexity: Learning Vector-Valued Functions
For certain families of multivariable vector-valued functions to be approximated, the accuracy of approximation schemes made up of linear combinations of computational units containing adjustable parameters is investigated. Upper bounds on the approximation error are derived that depend on the Rademacher complexities of the families. The estimates exploit possible relationships among the compon...
متن کامل